Learning Blackboard-Based Scheduling Algorithms for Computer Vision

The goal of image understanding by computer is to identify objects in visual images and (if necessary) to determine their location and orientation. Objects are identi ed by comparing data extracted from images to an a priori description of the object or object class in memory. It is a generally accepted premise that, in many domains, the timely and appropriate use of knowledge can substantially reduce the complexity of matching image data to object descriptions. Because of the variety and scope of knowledge relevant to di erent object classes, contexts and viewing conditions, blackboard architectures are well suited to the task of selecting and applying the relevant knowledge to each situation as it is encountered. This paper reviews ten years of work on the UMass VISIONS system ([12, 13]) and its blackboard-based high-level component, the schema system ([6, 7]). The schema system could interpret complex natural scenes when given carefully crafted knowledge bases describing the domain, but its application in practice was limited by the problem of model (knowledge base) acquisition. Experience with the schema system convinced us that learning techniques must be embedded in vision systems of the future to reduce or eliminate the knowledge engineering aspects of system construction. The Schema Learning System (SLS) is a supervised learning system for acquiring knowledge-directed object recognition (control) strategies from training images. The recognition strategies are precompiled reactive sequences of knowledge source invocations that replace the dynamic scheduler found in most blackboard systems. Each strategy is specialized to recognize instances of a speci c object class within a speci c context. Since the strategies are learned automatically, the knowledge base contains only general-purpose knowledge sources rather than problem-speci c control heuristics or sequencing information. This work was supported by DARPA under TACOM contract DAAE07-91-C-R035, RADC under contract F30602-91-C-0037 and the NSF under grant CDA-8922572.